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(57) Abstract 

An error is identified in a video image by calculating frequency-related coefficients for blocks of the image and considering the 
statistical distribution of said coefficients. The variance of the coefficients is calculated and a block of original image data is concealed 
if the variance calculated is significantly higher than a stored threshold and/or if said variance calculated is significantly different from 
variance values derived for similarly positioned blocks within a previous frame. Concealment is effected by selecting image data from a 
previous frame, displaced in response to transmitted motion vectors. 
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n^TBCTING ERRORS IN VIDEO IMAGES 

The present invention relates to detecting errors in 
signals representing video images. 
5 In addition to the broadcasting and storage of full 

bandwidth video signals, interest has also developed in 
techniques for reducing the bandwidth requirement for the 
transmission or storage of a video signal. These techniques 
facilitate, amongst other things, two-way transmission of 

10 video signals, leading to the development of affordable video 
conferencing facilities and video telephones. 

A problem with compressing video data for transmission 
over limited bandwidth transmission paths, is that the data 
becomes more susceptible to the presence of transmission 

15 errors. The nature of the transmission path will be such 
that errors usually cannot be prevented and in order to 
improve the overall result of transmitted images, it would be 
preferable to identify the presence of errors, and their 
locations within the video image and, where possible, take 

20 action to conceal these errors. 

One method for detecting errors comprises including an 
error correction code as an overhead. Cyclic redundancy 
codes are widely used for error detection at the expense' of 
a small overhead in the transmitted signal. 

25 If the error correction code is corrupted during 

transmission, this is detected at a receiver and the 
corrupted data replaced, for instance as described in US 
patent no. 5150210. 

The pres ent invention seeks to provide an improved 

30 method and apparatus for detecting the presence of errors in 
a video signal. 

According to a first aspect of the present invention, 
there is provided a method of detecting an error in a signal 
representing a video image, comprising: 

35 receiving blocks of frequency related coefficients 

representing respective blocks in the video image; and 
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analysing the statistical distribution of said coeffi- 
cients within a block to detect the presence of corrupted 
data. 

In a preferred embodiment, said analysing step 
5 involves calculating the variance of said coefficients in a 
block. Variance is determined by calculating the difference 
between actual coefficient values and the mean value, forming 
the square of said difference and summing said squares, i. e. 



N 

a (x^x) 2 

i=l 



10 

where o = variance 

Xj = value of the ith coefficient 

x = mean value of all the coefficents within a block 
N = number of coefficients in a block 

15 The variance may be compared against a predetermined 

threshold and an error identified if the variance is greater 
than said threshold. Alternatively or additionally, the 
variance values for the blocks of a frame may be stored and 
the variance for a block under consideration may be compared 

20 with the variance of the block in a previous frame that is 
similarly positioned and/or for the surrounding blocks of the 
previous frame. A mean variance value may be calculated for 
the blocks of the previous frame and an error detected if the 
variance value under consideration is significantly larger or 

25 significantly smaller than the mean value for the previous 
frame. 

If the transmission coding scheme employs frequency 
related coefficients which represent the actual values of the 
pixels within a block, for example the video signal is 
30 intraframe coded using DCT coefficients only, the 
mathematical distribution of the transmitted coefficients may 
be directly analysed. However the coding scheme employed 
might not result in frequency related coefficients 
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representing the actual value of the pixels. For instance, 
if the transmitted signal represents the difference between 
the current frame and an earlier frame, the actual pixel 
values, and hence the mathematical distribution of the 
5 resulting frequency coefficients, cannot be obtained without 
first decompressing the received signal to produce the actual 
pixel values, and then dividing the signal into blocks and 
transforming the signal into the frequency domain to produce 
the frequency related coefficients. 
10 In a preferred embodiment, action is taken to conceal 

a block if an error is identified. This may involve the 
substitution of the detected block by a similarly positioned 
block in a previous frame or its motion compensated block. 
Alternatively, a new motion vector may be calculated by 
15 averaging motion vectors for the similarly positioned block 
and surrounding blocks from a previous frame to produce a 
motion vector relating to a block in the previous frame with 
which the current * block is to be replaced. 

The invention will now be described by way of example 
20 only, with reference to the accompanying drawings, of which: 

Figure 1 shows a system for transmitting and receiving 
compressed video data, including a circuit for detecting and 
concealing errors ; 

Figure 2 shows how an image is made up from a plural- 
25 ity of blocks; 

Figure 3 shows a first embodiment of a circuit for 
detecting and concealing errors for use in a system as shown 
in Figure 1, including a processing unit; 

Figure 4 details the operation of the processing unit 
30 shown in Figure 3, for detecting the presence of an error; 

Figure 5 details the operations of the processing unit 
shown in Figure 3, for concealing an error. 

Figure 6 shows a system including a second embodiment 
of apparatus for detecting and concealing errors; 
35 Figure 7 shows in greater detail the apparatus shown 

in Figure 6 for detecting and concealing errors. 
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A system for transmitting compressed video data over 
a transmission path 15 from a transmitting station 16 to a 
receiving station 17 is shown in Figure 1. A conventional 
video signal is generated by a video source 18, which may be 
5 a video camera, a video tape player or similar equipment. 
The transmission path 15 has a limited bandwidth, therefore 
video data is compressed so as to be retained within the 
available bandwidth. The transmission path may be a 

telephone line, a dedicated digital link, a radio link or any 

10 other known means of providing a communication channel. By 
convention, the compressed video signal is in non-interlaced 
form, with each video frame having 288 lines with 352 picture 
elements on each line. In non-compressed form, each picture 
element has a luminance value represented by eight bits of 

15 data, with a smaller number of bits allocated to represent 
each colour difference signal. 

The video data generated by the video source 18 is 
compressed by a compression circuit 19 which may, for 
example, compress the video data in accordance with the CCITT 

20 H. 261 compression recommendation although the invention is 
not limited to this form of compression. According to the 
H. 261 recommendation, the video signal to be compressed is 
divided into portions representing blocks of pixels of the 
video image. Each block is transformed into the frequency 

2 5 domain by a discrete cosine transform (DCT) and the 
coefficients are transmitted. The blocks may be compressed 
without reference to any other block or frame (intra frame 
coding) or with reference to another block or frame, in which 
case the DCT coefficients represent the differences between 

30 the compared blocks. The original video data includes eight 
bits luminance for each picture element location and the 
block consists of an 8 x 8 array of picture elements. An 
array of coefficients is similarly proportioned but the 
resolution is such that a minimum of twelve bits may be 

35 required for a particular coefficient, plus a sign bit. 
Compression is achieved because many of the coefficients will 
have values of zero and may, therefore, effectively be 



BNSDOCID: <WO 9503674A1 _l_> 



WO 95/0367< 




PCT/GB94/01558 



- 5 - 



ignored. An output circuit 20 amplifies and, where required, 
modulates the compressed video signal, thereby placing it in 
a form suitable for transmission over the transmission path 
15. 

5 It is likely that the level of attenuation suffered by 

the transmitted signal will be frequency dependent, therefore 
an equalisation circuit 26 at the receiving station provides 
compensation. The received signal is then de-modulated (if 
required) by means not shown and amplified by an amplifying 

10 circuit 25. 

De-compression is performed by a de-compression 
circuit 27, arranged to perform the reverse process to the 
compression provided by the compression circuit 19. Any 
errors introduced into the signal, due to noise on the 

15 transmission channel 15, may result in corruption ofr the 
data. This would result in the corrupted data being visible 
on a display device 28. Thus, the overall integrity of the 
displayed image would be improved if the corrupted data could 
be detected and concealed in some way. 

20 To provide such a detection and concealing process, 

the system includes an error detecting and concealing circuit 
29, arranged to identify a block of corrupted data and to 
conceal this block of corrupted data by selecting an 
equivalent block from a previously transmitted frame. 

25 A conventional video frame 31, as shown in Figure 2, 

consists of 288 lines with 352 elements on each line. As 
part of the H. 261 compression procedure, the frame is divided 
into 1584 blocks, with sixty four picture elements, in the 
form of an 8 x 8 matrix, within each block. 

30 A luminance block 32 is shown in Figure 2 and this, in 

combination with its three adjacent blocks, provides a macro 
block 32. The macro block 33 is shown enlarged, with block 
34 displaying a full matrix of picture elements 35. In 
addition, a full colour picture also requires the trans - 

35 mission of two colour difference blocks per luminance block. 

The error detecting and concealing circuit 29 is 
detailed in Figure 3, in which decompressed video data from 
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the decompression circuit 27 is received at an input port 4 1 
and processed video data for display on the display device 28 
is applied to an output port 42. The circuit 29 includes a 
first image store 43 and a second image store 44, each 
5 capable of storing a full video frame. A video write 
controller 45 controls the writing of video data to the image 
stores, such that, a first frame is written to image store 43 
and a second frame is written to store 44, while the first 
frame is being read from the first image store 43 under the 

10 control of a video read controller 46. After a full video 
frame has been written to the second image store 44, the next 
video frame is written to the first image store 4 3, 
overwriting the previously written frame and an output signal 
for port 42 is derived by reading the image from store 44. 

15 While data is being written to one of the image stores 

43 or 44, said data is also processed to detect the presence 
of errors. When an error is detected, blocks of data in the 
image stores 4 3 and 44 may be overwritten, under the control 
of a block overwrite controller 47. In order to identify the 

20 presence of errors, the input image data from port 41 is also 
supplied to a transform unit 48, arranged to transform the 
input image data into frequency related coefficients for each 
block of picture elements. In the preferred embodiment, the 
transform unit performs a discrete cosine transform (DCT) on 

25 the blocks of image data. The frequency related coefficients 
are then supplied to a processing unit 49. 

In order to detect the presence of errors in the 
transmitted video data, the processing unit 49 is arranged to 
calculate the mean and variance of the coefficient values 

30 within each block. As these variance values are calculated, 
they are supplied to a first variance store 55 or to a second 
variance store 56, thereby ensuring that variance values 
calculated for the previous frame are available to the 
processing unit 49. The writing and reading of variance 

35 values to and from stores 55 and 56 is controlled by a 
variance store controller 57. 
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If the video information supplied over the trans- 
mission path 15 is compressed in a form such that, in 
addition to including spatially compressed coefficients, data 
representing motion vectors for each block of compressed data 
5 are also supplied, the motion vectors are also supplied to 
the error concealing circuit 29 via an input port 50. 

Motion vectors are calculated by comparing a block of 
picture elements in a current frame with a similarly 
positioned block in a previous frame and with blocks, shifted 

10 by a plurality of picture element displacements in both the 
x and y directions. The motion vector is not related 
directly to movement of objects within the original image but 
actually represents the closest fit, derived by comparing the 
block of interest with similar blocks of the previous frame. 

15 A technique for performing such comparisons in order to 
produce motion vectors, is disclosed in United States Patent 
No. 5, 083, 202, assigned to the present applicant. 

Thus, for e.ach block of video data, x and y values are 
transmitted indicating a motion vector of the closest fitting 

20 block from the previous frame. These displacement vectors 
are supplied to a vector-store write-controller 51, wherein 
vectors derived from a first frame are written to a first 
vector store 52, vectors from the next frame are written to 
a second vector store 53, whereafter the first store is over- 

25 written etc. Thus, vector values for the previous frame are 
available to the processing unit 49, via a vector store 
reading circuit 54. 

Operational procedures for the processing unit 49 are 
detailed in Figure 4. As a result of the transform performed 

30 by transform unit 48, a frame of coefficients will become 
available to processing unit 49, which initiates its 
processing procedures at step 6 L A question is asked at 
step 62 as to whether another block of the frame is available 
and, for the first block of a frame, this question will be 

35 answered in the affirmative. When answered in the affirm- 
ative, the mean value for the coefficients in the block is 
calculated at step 63. The mean value for the coefficients 
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of an 8x8 block is derived by adding the values of the 
coefficients together and then dividing by 64. 

At step 64 the variance of the values is calculated by 
subtracting the mean value from each coefficient value to 
5 produce a difference value for each particular coefficient. 
This difference value is squared and the variance is obtained 
by adding all 64 squared terms. 

At step 65 the variance value calculated for the 
particular block is stored in variance store 55 or 56, 

10 depending upon the phase of the particular frame under 
consideration. 

At step 66 a threshold value T is read and at step 67 
a question is asked as to whether the variance value 
calculated at step 64 is larger than the threshold value T. 

15 The threshold value T is adjustable or selectable by an 
operator and may be adjusted to suit a particular type of 
video transmission. If the variance value calculated at step 
64 is larger than the threshold value read at step 66, it is 
assumed that the block under consideration contains errors, 

20 in that a large variance value has been produced due to the 
presence of errors. Thus, if the question asked at step 67 
is answered in the affirmative, the block is concealed by 
invoking a conceal block routine at step 68. 

If the question asked at step 67 is answered in the 

25 negative, a further check is performed on the variance value 
to determine whether said value represents the presence of an 
error. Previously, said variance value was compared against 
a threshold value, which is appropriate for identifying very 
severe errors which produce very large variance values. 

30 However, a block having coefficients with a modest variance 
may still be in error and such an error is detected if the 
variance is significantly different from the variance values 
of blocks surrounding the block under consideration, in the 
equivalent position of a previous frame. 

35 Thus, at step 69, the processing unit accesses the 

variance store for the previous frame. Therefore, if the 
variance value calculated at step 64 was written to store 55, 
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step 69 accesses variance values from store 56. The 
equivalent position to the block under consideration is 
identified and the variance values for it and the eight 
surrounding blocks are read from store 5 6. 
5 At step 70 the mean value P for the previous frame 

variance values is calculated and a comparison of this 
previous mean value is made with the present variance value, 
at step 71. If the value for the block under consideration 
is greater than three times the previous mean value P or 

10 smaller than the previous mean value P divided by three, it 
is assumed that the block contains an error and the 
concealing algorithm is again invoked. Thus, if the value is 
greater than three times the previous mean or smaller than 
said previous mean divided by three, the question asked at 

15 step 71 is answered in the affirmative and the conceal block 
routine is called at step 72. Alternatively, if the question 
asked at step 71 is answered in the negative, the block is 
considered to be- error free at step 73 and control is 
returned to step 62. Eventually, all of the blocks for a 

20 particular frame will have been considered and the question 
asked at step 62 will be answered in the negative, returning 
control to step 61 and placing the processing unit 49 in a 
state ready for the next frame of coefficients. 

The concealing routine which may be called at step 68 

25 or at step 72 is detailed in Figure 5. For the purposes of 
this example, it is assumed that image data is being written 
to image store 43 and that the processing unit 49 has 
identified a block of image data which contains an error. As 
data is written to image store 43, previously processed data 

30 is read from image store 44, thereby providing a video output 
signal to output port '42. A period of time is therefore 
available during which modifications may be made to the image 
data stored in store 43, before said data is selected by the 
output controller 46. As image data is written to store 43, 

35 motion vectors are written to vector store 52 and, similarly, 
as the writing of image data is switched to image store 44, 
the writing of motion vector data is switched to store 53. 



BNSDOCID: <WO 9503674A1 J_> 



WO 95/03674 



f 

!T/GB94/01558 



Thus, an error is detected by the processing unit in 
a block of image data which has been written to the image 
store 43 and the processing unit 49 is now required to effect 
procedures to conceal the error before this data is supplied 
5 to the output port 42. 

At step 81 of Figure 5, the vector store 52 is 
accessed so as to read the motion vector for the equivalent 
block of the previous frame. 

At step 82 the motion vectors for the eight surround- 
10 ing blocks are read from vector store 52, thereby providing 
a total of nine motion vectors to the processing unit 49. 

At step 83 an average motion vector is calculated by 
adding said nine values and dividing by nine to produce an 
averaged motion vector for accessing image data of the 
15 previous frame. Thus, the average motion vector identifies 
the position of a block in the previous frame which, after 
being moved in the x and y directions by amounts specified by 
the motion vector, provides a close match to the block under 
consideration in the present frame. 
20 Thus, the averaged motion vector identifies a block of 

data in the previous frame which, at this point in time, will 
be held in image store 44, the store presently being read to 
provide an output signal. Thus, at step 85 the data 
identified in store 44 is read by the block overwrite 
25 controller 47, in response to instructions received from the 
processing unit 4 9, and written to the block under 
consideration in the input image store 43. 

It is important to note that the block read from the 
output image store 44 will not necessarily lie within an 
30 original block boundary, given that the motion vectors are 
specified for picture element positions. 

After the image block has been overwritten, control is 
returned to step 62, allowing another block. to be considered. 

Figure 6 shows a system in which the compressed 
35 transmitted signal comprises frequency related coefficients 
that represent the actual pixel values of the frame. Little 
or no further processing of the DCT coefficients is therefore 
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required before they are input to the processing unit 49, as 
shown in Figure 7. The operation of the circuit as shown in 
Figure 7 is otherwise the same as that shown in Figure 3. 

The decompression circuit 27 may include some 
5 conventional form of error checking, for instance error 
correction code checking means. In this case, the 

decompression circuit flags a macroblock or a Group of blocks 
(GOB) that is identified as containing an error, (a Group of 
blocks comprises a matrix of 11 macroblocks by 3 

10 macroblocks). Only those blocks of a flagged macroblock or 
GOB are passed to the error detecting and concealing circuit 
29 to determine which block within the macroblock or GOB 
contains an error. Those blocks that are not corrupted may 
therefore be retained, whereas those blocks in which an error 

15 is detected can be concealed. 
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1. A method of detecting an error in a signal 

representing a video image, comprising: 

receiving blocks of frequency related coefficients 
5 representing respective blocks of the image; and 

analysing the statistical distribution of said coeffi- 
cients within a block to detect the presence of corrupted 
data. 

10 2. A method according to claim 1, wherein said analysing 

step involves calculating the variance of said coefficients 
in a block. 

3. A method according to claim 2, further comprising 

15 comparing the calculated variance of a block with a predeter- 
mined threshold value, an error being detected if the 
calculated variance exceeds the predetermined threshold 
value. 

20 4. A method according to claim 2 or 3 further comprising 

comparing the calculated variance of a block of a current 
frame with the variance value(s) calculated for at least one 
similarly positioned block(s) in a previous frame, an error 
being detected if the calculated variance for a block of the 

25 currenr frame differs from the variance of the similarly 
positioned block(s) of a previous frame by a predetermined 
amount. 

5. A method according to any preceding claim further 

30 comprising firstly dividing the signal into a plurality of 
portions corresponding to respective blocks (32) of an image; 
and transforming said portions into blocks of frequency 
related coefficients. 

35 6. Apparatus for identifying the presence. of an error in 

a signal representing a video image, characterised by: 
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means (49) for analysing the statistical distribution 
of frequency related coefficients representing respective 
blocks of the image to- identify corrupted data. 

5 7. Apparatus according to claim 6, wherein said analysing 

means includes means for calculating the variance of the 
coefficients in a block. 

8. Apparatus according to claim 7, wherein said analysing 
10 means includes means for comparing said variance value 

against a predetermined threshold value. 

9. Apparatus according to claim 7 or 8, wherein said 
analysing means includes means for comparing said variance 

15 value against variance value(s) obtained for at leasts one 
similarly positioned block(s) in a previous frame. 

10. Apparatus according to any one of claims 6 to 9, 
including means for storing image data and means operable in 

20 response to detection of an error within a block to 
substitute for that block a stored block from an earlier 
frame of the image. 

11. Apparatus according to claim 10, including means (52, 
25 53/ 54) for reading motion vectors included with transmitted 

video data, and processing means (49, 47) for concealing 
blocks identified as containing errors by selecting a stored 
block in response to said motion vectors. 

30 12. Apparatus according to claim 11, wherein a new motion 
vector is calculated by averaging the motion vectors 
associated with similarly positioned blocks within the 
previous frame. 

35 13. Apparatus according to any of claims 6 to 12 further 
comprising: 



BNSDOCID: <WO 950367 4A1_I_> 



WO 95/03674 ^ ^T7GB94/01558 

- 14 - 

means (48) for firstly dividing a signal into a 
plurality of portions corresponding to respective blocks (32) 
of an image; and 

means (48) for transforming said portions into blocks 
5 of frequency related coefficients 
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